In person and virtual process mapping experiences to capture and explore variability in clinical practice: application to genetic referral pathways across seven Australian hospital networks.

Genetic referral for Lynch syndrome (LS) exemplifies complex clinical pathways. Identifying target behaviours (TBs) for change and associated barriers requires structured group consultation activities with busy clinicians - consolidating implementation activities whilst retaining rigour is crucial. This study aimed to: i) use process mapping to gain in-depth understandings of site-specific LS testing and referral practices in Australian hospitals and support identification of TBs for change, ii) explore if barriers to identified TBs could be identified through process mapping focus-group data, and iii) demonstrate pandemic-induced transition from in-person to virtual group interactive process mapping methods. LS clinical stakeholders attended interactive in-person or virtual focus groups to develop site-specific "process maps" visually representing referral pathways. Content analysis of transcriptions informed site-specific process maps, then clinical audit data was compared to highlight TBs for change. TBs were reviewed in follow-up focus groups. Secondary thematic analysis explored barriers to identified TBs, coded against the Theoretical Domains Framework (TDF). The transition from in-person to pandemic-induced virtual group interactive process mapping methods was documented. Process mapping highlighted six key areas of clinical practice variation across sites and site-specific TBs for change were identified. Key barriers to identified TBs emerged, categorised to seven TDF domains. Process mapping revealed variations in clinical practices surrounding LS referral between sites. Incorporating qualitative perspectives enhances process mapping by facilitating identification of TBs for change and barriers, providing a pathway to developing targeted interventions. Virtual process mapping activities produced detailed data and enabled comprehensive map development.

To achieve change in the health system using implementation approaches, time-poor clinicians must engage in information-gathering and idea-generation activities. This research revealed that qualitative process mapping focus groups held both in-person and virtually can be used to streamline these activities, by simultaneously identifying target behaviours for change, and barriers to change, while gaining information about site-specific clinical processes. Hospital process mapping shows that complex clinical processes vary significantly between sites, and that understanding local variation is crucial to developing targeted interventions. This study has informed new approaches to implementation research methods.

Lynch syndrome testing of colorectal cancer patients in a high-income country with universal healthcare: a retrospective study of current practice and gaps in seven australian hospitals.

BACKGROUND: To inform effective genomic medicine strategies, it is important to examine current approaches and gaps in well-established applications. Lynch syndrome (LS) causes 3-5% of colorectal cancers (CRCs). While guidelines commonly recommend LS tumour testing of all CRC patients, implementation in health systems is known to be highly variable. To provide insights on the heterogeneity in practice and current bottlenecks in a high-income country with universal healthcare, we characterise the approaches and gaps in LS testing and referral in seven Australian hospitals across three states. METHODS: We obtained surgery, pathology, and genetics services data for 1,624 patients who underwent CRC resections from 01/01/2017 to 31/12/2018 in the included hospitals. RESULTS: Tumour testing approaches differed between hospitals, with 0-19% of patients missing mismatch repair deficiency test results (total 211/1,624 patients). Tumour tests to exclude somatic MLH1 loss were incomplete at five hospitals (42/187 patients). Of 74 patients with tumour tests completed appropriately and indicating high risk of LS, 36 (49%) were missing a record of referral to genetics services for diagnostic testing, with higher missingness for older patients (0% of patients aged ≤ 40 years, 76% of patients aged > 70 years). Of 38 patients with high-risk tumour test results and genetics services referral, diagnostic testing was carried out for 25 (89%) and identified a LS pathogenic/likely pathogenic variant for 11 patients (44% of 25; 0.7% of 1,624 patients). CONCLUSIONS: Given the LS testing and referral gaps, further work is needed to identify strategies for successful integration of LS testing into clinical care, and provide a model for hereditary cancers and broader genomic medicine. Standardised reporting may help clinicians interpret tumour test results and initiate further actions.

Building capacity from within: qualitative evaluation of a training program aimed at upskilling healthcare workers in delivering an evidence-based implementation approach.

Translating evidence into complex health systems is an ongoing challenge. Building the capacity of healthcare workers in behavioral and implementation science methods may facilitate the use of evidence-based implementation approaches, leading to sustainable and effective translation. The aim was to describe the development, contents and evaluation of a training workshop aimed at upskilling hospital-embedded staff to deliver an evidence-based implementation approach. The Hide and Seek Project (HaSP) is a cluster randomized controlled trial testing two implementation approaches for improving hereditary cancer referral at eight Australian hospitals. Healthcare workers were recruited as "Implementation Leads" and trained via a one-day workshop-TRAining in evideNce-baSed ImpLementATion for hEalth (TRANSLATE). The purpose of TRANSLATE was to upskill Implementation Leads in the delivery of HaSP, as well as implementation science methods more broadly. Implementation Leads participated in semi-structured evaluation interviews, which were analyzed using inductive thematic analysis. Nine Implementation Leads from various professional backgrounds completed the training. Four key themes were identified: (i) training day reactions, (ii) learning, (iii) implementation barriers and facilitators, and (iv) building health system capacity for implementation. Participants reported high levels of satisfaction, and anticipated that the knowledge and skills may be useful in the future. We describe a novel training program focused on the delivery of evidence-based implementation within health systems. Guided by insights from this study, methods to deliver the training on a larger scale and across different contexts are being explored. The prolonged impact of TRANSLATE will be further evaluated at trial completion. Trial registration: ANZCTR, ACTRN12618001072202. Registered on June 27, 2018.

What would I do? Perspectives on the factors underlying Lynch syndrome genetic testing and results sharing decisions for high-risk colorectal cancer patients.

OBJECTIVE: Universal tumour testing for Lynch syndrome (LS) in all incident colorectal cancers (CRCs) and sequential diagnostic genetic testing is cost-effective in Australia. Because of this, our study aimed to understand factors underlying possible decisions faced by tumour test-positive CRC patients and their at-risk relatives throughout the LS diagnosis pathway. METHODS: Semi-structured telephone interviews were conducted with 23 participants, using four hypothetical scenarios. Vignette-guided closed- and open-ended questions asked about LS genetic testing uptake, discussing diagnosis with at-risk relatives, and risk-reducing interventions. Personal perspectives on genetic testing were collected pre-post vignette discussion. Inductive thematic analysis was performed on open-ended questions. Decisional pathway diagrams were developed to convey factors influencing complex decision-making processes. RESULTS: Participant responses incorporated unfolding scenario information, resulting in three decision themes: (1) wanting to know one's LS status; (2) informing family about LS; (3) navigating risk-reducing interventions. Across all themes, 'knowledge' emerged as a facilitator, and 'negative emotional experience' as a barrier. Personal supportive views toward genetic testing increased post-interview. CONCLUSIONS: When communicating with tumour test-positive CRC patients or their relatives about LS genetic testing, providing guidance/resources to inform decisions around risk-reducing interventions and informing family members is critical. Scenario-driven interviews provide insight into what individuals might do when facing complex healthcare decisions and could aid informed decision-making. This approach may be applicable in other conditions, particularly with mainstreaming being increasingly introduced into the genetic context.

Do organization-level quality management systems influence department-level quality? A cross-sectional study across 32 large hospitals in Australia.

OBJECTIVE: Little is known about the influence that hospital quality systems have on quality at department level, in Australia and elsewhere. This study assessed the relationships between organizational-level quality management systems, and the extent to which hospital-level quality management systems and department-level quality management strategies are related. DESIGN: A multi-level, cross-sectional, mixed-method study. SETTING AND PARTICIPANTS: As part of the Deepening our Understanding of Quality in Australia (DUQuA) project, we invited all large hospitals in Australia (~200 or more beds) which provided acute myocardial infarction (AMI), hip fracture and stroke care. The quality managers of these hospitals were the respondents for one of seven measures of hospital quality management systems and strategies. Data across the six remaining measures were collected through site visits by external surveyors assessing the participating hospitals. MAIN OUTCOME MEASURES: Relationships were assessed between three organization-level quality management system measures: a self-report measure assessing organization-level quality activities (quality management systems index, QMSI); externally assessed organization-level compliance to procedures used to plan, monitor and improve quality of care (quality management compliance index, QMCI); and externally assessed implementation of quality systems (clinical quality implementation index, CQII). Associations were also assessed between organization-level quality management systems and department-level quality management strategies: how clinical responsibilities are assigned for a particular condition; whether department organization processes are organized to facilitate evidence-based care recommendations; compliance with selected recommendations of international agencies; and whether clinical reviews are performed systematically. RESULTS: Of 78 invited hospitals, 32 participated in the study. QMSI was positively associated with QMCI and CQII, but after controlling for QMSI, no relationship was found between QMCI and CQII. There appears to be a cluster of relationships between QMSI and department-level measures, but this was not consistent across all departments. CONCLUSION: This is the first national study undertaken in Australia to assess relationships within and between organization-level and department-level quality management systems. These quality management system tools align with many components of accreditation standards and may be useful for hospitals in continuously monitoring and driving improvement.

Using accreditation surveyors to conduct health services research: a qualitative, comparative study in Australia.

OBJECTIVE: Healthcare accreditation surveyors are well positioned to gain access to hospitals and apply their existing data collection skills to research. Consequently, we contracted and trained a surveyor cohort to collect research data for the Deepening our Understanding of Quality in Australia (DUQuA) project. The aim of this study is to explore and compare surveyors' perceptions and experiences in collecting quality and safety data for accreditation and for health services research. DESIGN: A qualitative, comparative study. SETTING AND PARTICIPANTS: Ten surveyors participated in semi-structured interviews, which were audio recorded, transcribed and coded using Nvivo11. Interview transcripts of participants were analysed thematically and separately, providing an opportunity for comparison and for identifying common themes and subthemes. INTERVENTION(S): None. MAIN OUTCOME MEASURE(S): Topics addressed data collection for healthcare accreditation and research, including preparation and training, structure, organization, attitudes and behaviours of staff and perceptions of their role. RESULTS: Five themes and ten subthemes emerged from the interviews: (1) overlapping facilitators for accreditation and research data collection, (2) accreditation-specific facilitators, (3) overlapping barriers for accreditation and research data collection, (4) research data collection-specific barriers and (5) needs and recommendations. Subthemes were (1.1) preparation and training availability, (1.2) prior knowledge and experiences; (2.1) ease of access, (2.2) high staff engagement, (3.1) time, (4.1) poor access and structure, (4.2) lack of staff engagement, (4.3) organizational changes; (5.1) short-notice accreditation and (5.2) preparation for future research. CONCLUSIONS: Although hospital accreditation and research activities require different approaches to data collection, we found that suitably trained accreditation surveyors were able to perform both activities effectively. The barriers surveyors encountered when collecting data for research provide insight into the challenges that may be faced when visiting hospitals for short-notice accreditation.

Validation of the patient measure of safety (PMOS) questionnaire in Australian public hospitals.

OBJECTIVE: Patients can provide a unique perspective on the safety of care in hospitals. Understanding that the extent to which the way hospitals are organized for quality and safety is associated with patient perceptions of care is becoming increasingly valued and necessary for the direction of targeted interventions across healthcare systems. The UK-developed patient measure of safety (PMOS) assesses eight domains of ward safety from the patient point of view and has recently been adapted and piloted in Australia. The aim of this study is to test the psychometric properties of PMOS-Australia (PMOS-A) amongst a large cohort of hospitalized patients. DESIGN: Cross-sectional questionnaire validation assessment. SETTING AND PARTICIPANTS: As part of the DUQuA project, the PMOS-A survey was distributed within acute myocardial infarction, hip fracture and stroke departments across 32 large public hospitals in Australia. Patients could complete the PMOS-A independently, or request the assistance of a family member/guardian, or staff on the wards-space was included to record mode of completion. MAIN OUTCOME MEASURES: Confirmatory factor analysis (CFA) was undertaken on a calibration sample to generate the model, and a validation sample was used to cross-validate the model. A subset of only those participants who received assistance for PMOS-A completion was also tested using CFA on a calibration and validation sample. Model fit indices (chi-square to degrees of freedom ratio [Chi-square:DF], root mean square error of approximation [RMSEA], comparative fit indices [CFI], standardized root mean squared residual [SRMR]), Cronbach's α, average inter-item correlations, construct reliability and cross-loadings were examined with reference to recommended thresholds to establish the extent of convergent validity and discriminant validity. A marker of criterion validity was assessed through testing associations between the PMOS-A and adherence to clinical guidelines. RESULTS: Across the calibration and validation samples of the full (N = 911) and assisted completers only subset (N = 490), three (Chi-square:DF, SRMR, RMSEA) of the four indices consistently or almost always met thresholds for acceptable model fit. CFI indices did not meet the recommended limits (0.72-0.78, against a target > 0.9). Positive relationships were found for all tests between PMOS-A and adherence to clinical guidelines, and these were significant when assessed in the calibration datasets for the full and assisted completion samples. CONCLUSION: A sufficiently reliable and valid measure of patient perceptions of safety has been developed. These findings should provide adequate support to justify the use of this measure to assess patient perceptions of safety in Australian hospitals and can be modified for use elsewhere.

Deepening our Understanding of Quality in Australia (DUQuA): An overview of a nation-wide, multi-level analysis of relationships between quality management systems and patient factors in 32 hospitals.

OBJECTIVE: The Deepening our Understanding of Quality in Australia (DUQuA) project is a multisite, multi-level, cross-sectional study of 32 of the largest hospitals in Australia. This overview examines relationships between (i) organization-level quality management systems and department-level quality management strategies and (ii) patient-level measures (clinical treatment processes, patient-reported perceptions of care and clinical outcomes) within Australian hospitals. DESIGN: We examined hospital quality improvement structures, processes and outcomes, collecting data at organization, department and patient levels for acute myocardial infarction (AMI), hip fracture and stroke. Data sources included surveys of quality managers, clinicians and patients, hospital visits, medical record reviews and national databases. Outcomes data and patient admissions data were analysed. Relationships between measures were evaluated using multi-level models. We based the methods on the Deepening our Understanding of Quality Improvement in Europe (DUQuE) framework, extending that work in parts and customizing the design to Australian circumstances. SETTING, PARTICIPANTS AND OUTCOME MEASURES: The 32 hospitals, containing 119 participating departments, provided wide representation across metropolitan, inner and outer regional Australia. We obtained 31 quality management, 1334 clinician and 857 patient questionnaires, and conducted 2401 medical record reviews and 151 external assessments. External data via a secondary source comprised 14 460 index patient admissions across 14 031 individual patients. Associations between hospital, Emergency Department (ED) and department-level systems and strategies and five patient-level outcomes were assessed: 19 of 165 associations (11.5%) were statistically significant, 12 of 79 positive associations (15.2%) and 7 of 85 negative associations (8.2%). RESULTS: We did not find clear relationships between hospital-level quality management systems, ED or department quality strategies and patient-level outcomes. ED-level clinical reviews were related to adherence to clinical practice guidelines for AMI, hip fracture and stroke, but in different directions. The results, when considered alongside the DUQuE results, are suggestive that front line interventions may be more influential than department-level interventions when shaping quality of care and that multi-pronged strategies are needed. Benchmark reports were sent to each participating hospital, stimulating targeted quality improvement activities. CONCLUSIONS: We found no compelling relationships between the way care is organized and the quality of care across three targeted patient-level outcome conditions. The study was cross-sectional, and thus we recommend that the relationships studied should be assessed for changes across time. Tracking care longitudinally so that quality improvement activities are monitored and fed back to participants is an important initiative that should be given priority as health systems strive to develop their capacity for quality improvement over time.

Comparing theory and non-theory based implementation approaches to improving referral practices in cancer genetics: a cluster randomised trial protocol.

BACKGROUND: Lynch syndrome (LS) is an inherited, cancer predisposition syndrome associated with an increased risk of colorectal, endometrial and other cancer types. Identifying individuals with LS allows access to cancer risk management strategies proven to reduce cancer incidence and improve survival. However, LS is underdiagnosed and genetic referral rates are poor. Improving LS referral is complex, and requires multisystem behaviour change. Although barriers have been identified, evidence-based strategies to facilitate behaviour change are lacking. The aim of this study is to compare the effectiveness of a theory-based implementation approach against a non-theory based approach for improving detection of LS amongst Australian patients with colorectal cancer (CRC). METHODS: A two-arm parallel cluster randomised trial design will be used to compare two identical, structured implementation approaches, distinguished only by the use of theory to identify barriers and design targeted intervention strategies, to improve LS referral practices in eight large Australian hospital networks. Each hospital network will be randomly allocated to a trial arm, with stratification by state. A trained healthcare professional will lead the following phases at each site: (1) undertake baseline clinical practice audits, (2) form multidisciplinary Implementation Teams, (3) identify target behaviours for practice change, (4) identify barriers to change, (5) generate intervention strategies, (6) support staff to implement interventions and (7) evaluate the effectiveness of the intervention using post-implementation clinical data. The theoretical and non-theoretical components of each trial arm will be distinguished in phases 4-5. Study outcomes include a LS referral process map for each hospital network, with evaluation of the proportion of patients with risk-appropriate completion of the LS referral pathway within 2 months of CRC resection pre and post implementation. DISCUSSION: This trial will determine the more effective approach for improving the detection of LS amongst patients with CRC, whilst also advancing understanding of the impact of theory-based implementation approaches in complex health systems and the feasibility of training healthcare professionals to use them. Insights gained will guide the development of future interventions to improve LS identification on a larger scale and across different contexts, as well as efforts to address the gap between evidence and practice in the rapidly evolving field of genomic research. TRIAL REGISTRATION: ANZCTR, ACTRN12618001072202 . Registered on 27 June 2018.

Older, vulnerable patient view: a pilot and feasibility study of the patient measure of safety (PMOS) with patients in Australia.

OBJECTIVES: The UK-developed patient measure of safety (PMOS) is a validated tool which captures patient perceptions of safety in hospitals. We aimed (1) to investigate the extent to which the PMOS is appropriate for use with stroke, acute myocardial infarction (AMI) and hip fracture patients in Australian hospitals and (2) to pilot the PMOS for use in a large-scale, national study 'Deepening our Understanding of Quality in Australia' (DUQuA). PARTICIPANTS: Stroke, AMI and hip fracture patients (n=34) receiving care in 3 wards in 1 large hospital. METHODS: 2 phases were conducted. First, a 'think aloud' study was used to determine the validity of PMOS with this population in an international setting, and to make amendments based on patient feedback. The second phase tested the revised measure to establish the internal consistency reliability of the revised subscales, and piloted the recruitment and administration processes to ensure feasibility of the PMOS for use in DUQuA. RESULTS: Of the 43 questions in the PMOS, 13 (30%) were amended based on issues patients highlighted for improvement in phase 1. In phase 2, a total of 34 patients were approached and 29 included, with a mean age of 71.3 years (SD=16.39). Internal consistency reliability was established using interitem correlation and Cronbach's α for all but 1 subscale. The most and least favourably rated aspects of safety differed between the 3 wards. A study log was categorised into 10 key feasibility factors, including liaising with wards to understand operational procedures and identify patterns of patient discharge. CONCLUSIONS: Capturing patient perceptions of care is crucial in improving patient safety. The revised PMOS is appropriate for use with vulnerable older adult groups. The findings from this study have informed key decisions made for the deployment of this measure as part of the DUQuA study.

Deepening our Understanding of Quality in Australia (DUQuA): a study protocol for a nationwide, multilevel analysis of relationships between hospital quality management systems and patient factors.

INTRODUCTION: Despite the growing body of research on quality and safety in healthcare, there is little evidence of the association between the way hospitals are organised for quality and patient factors, limiting our understanding of how to effect large-scale change. The 'Deepening our Understanding of Quality in Australia' (DUQuA) study aims to measure and examine relationships between (1) organisation and department-level quality management systems (QMS), clinician leadership and culture, and (2) clinical treatment processes, clinical outcomes and patient-reported perceptions of care within Australian hospitals. METHODS AND ANALYSIS: The DUQuA project is a national, multilevel, cross-sectional study with data collection at organisation (hospital), department, professional and patient levels. Sample size calculations indicate a minimum of 43 hospitals are required to adequately power the study. To allow for rejection and attrition, 70 hospitals across all Australian jurisdictions that meet the inclusion criteria will be invited to participate. Participants will consist of hospital quality management professionals; clinicians; and patients with stroke, acute myocardial infarction and hip fracture. Organisation and department-level QMS, clinician leadership and culture, patient perceptions of safety, clinical treatment processes, and patient outcomes will be assessed using validated, evidence-based or consensus-based measurement tools. Data analysis will consist of simple correlations, linear and logistic regression and multilevel modelling. Multilevel modelling methods will enable identification of the amount of variation in outcomes attributed to the hospital and department levels, and the factors contributing to this variation. ETHICS AND DISSEMINATION: Ethical approval has been obtained. Results will be disseminated to individual hospitals in de-identified national and international benchmarking reports with data-driven recommendations. This ground-breaking national study has the potential to influence decision-making on the implementation of quality and safety systems and processes in Australian and international hospitals.

High performing hospitals: a qualitative systematic review of associated factors and practical strategies for improvement.

BACKGROUND: High performing hospitals attain excellence across multiple measures of performance and multiple departments. Studying high performing hospitals can be valuable if factors associated with high performance can be identified and applied. Factors leading to high performance are complex and an exclusive quantitative approach may fail to identify richly descriptive or relevant contextual factors. The objective of this study was to undertake a systematic review of qualitative literature to identify methods used to identify high performing hospitals, the factors associated with high performers, and practical strategies for improvement. METHODS: Methods used to collect and summarise the evidence contributing to this review followed the 'enhancing transparency in reporting the synthesis of qualitative research' protocol. Peer reviewed studies were identified through Medline, Embase and Cinahl (Jan 2000-Feb 2014) using specified key words, subject terms, and medical subject headings. Eligible studies required the use of a quantitative method to identify high performing hospitals, and qualitative methods or tools to identify factors associated with high performing hospitals or hospital departments. Title, abstract, and full text screening was undertaken by four reviewers, and inter-rater reliability statistics were calculated for each review phase. Risk of bias was assessed. Following data extraction, thematic syntheses identified contextual factors important for explaining success. Practical strategies for achieving high performance were then mapped against the identified themes. RESULTS: A total of 19 studies from a possible 11,428 were included in the review. A range of process, output, outcome and other indicators were used to identify high performing hospitals. Seven themes representing factors associated with high performance (and 25 sub-themes) emerged from the thematic syntheses: positive organisational culture, senior management support, effective performance monitoring, building and maintaining a proficient workforce, effective leaders across the organisation, expertise-driven practice, and interdisciplinary teamwork. Fifty six practical strategies for achieving high performance were catalogued. CONCLUSIONS: This review provides insights into methods used to identify high performing hospitals, and yields ideas about the factors important for success. It highlights the need to advance approaches for understanding what constitutes high performance and how to harness factors associated with high performance.